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Abstract 

The Shannon-Khinchin axioms are generalized to nonextensive systems and the uniqueness the- 
orem for the nonextensive entropy is proved rigorously. In the present axioms, Shannon additivity 
is used as additivity in contrast to pseudoadditivity in Abe's axioms. The results reveal that 
Tsallis entropy is the simplest among all nonextensive entropies which can be obtained from the 
generalized Shannon-Khinchin axioms. 
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I. INTRODUCTION 



Tsallis entropy, which was first introduced by Tsallis in 1988 Jl|], is defined by 

Sj (p 1 ,...,p n ) = , (1) 

for a given probability distribution {pi}i=i... n , where n is a number of accessible configura- 
tions, q is a real parameter assumed to be positive, and Boltzmann's constant k has been 
set equal to unity 

Tsallis entropy is a one-parameter generalization of Shannon entropy in the sense that 

n 

limSg = Si^-^pilnpi. (2) 

i=l 

The characteristic property of Tsallis entropy is called pseudoadditivity, i.e. 

Sj (A, B) = Sj (A) + S T q (B) + (1 - q) S T q (A) Sj (B) (3) 
holds true for two mutually independent finite event systems A and B 

A - ( Al "' A A B - ( Bl "' (4) 

such that pfj B = pfpf for any i = 1, . . . , n and j = 1, . . . , m. (5) 

The property (f|) represents nonextensive additivity of Tsallis entropy, i.e. Tsallis entropy 
is a nonextensive entropy. When q = 1 the pseudoadditivity @ reverts to the standard 
additivity, so that Shannon entropy is considered to be extensive entropy in contrast to 
Tsallis entropy. 

Introduction of Tsallis entropy opened a new research area in statistical physics, provid- 
ing an advantageous generalization of traditional Boltzmann-Gibbs statistical mechanics || . 
The generalization enables us to find a consistent treatment of dynamics in many nonex- 
tensive physical systems such as long-range interactions, long-time memories, and multi- 
fractal structures, which cannot be coherently explained within the conventional Boltzmann- 
Gibbs statistics 0. Thus Tsallis entropy inspires many physicists to establish a generalized 
Boltzmann-Gibbs statistical mechanics leading to numerous applications P, E3J. 
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Along with the rapid progress of research in the generalized Boltzmann-Gibbs statistical 
mechanics, some axioms of Tsallis entropy ([!]) were presented as generalization of Shannon- 
Khinchin axioms [|], |^J. We proceed with a discussion of the Shannon-Khinchin axioms and 
their possible generalized counterparts. 

Let A n be defined by the n- dimensional simplex: 



n 

>0, ^p l = l}>. (6) 



P 

i=i 



An=|(Pl,---,P 

The Shannon-Khinchin axioms || are given by the following four conditions: 

[SKI] continuity: for any n G N the function Si (p) is continuous with respect to p G A n , 

[SK2] maximality: for given n G N and for (p 1; . . . , p n ) G A n , the function Si (pi, . . . , p n ) 
takes its largest value for p^ = - (i = 1, . . . , n), 

[SK3] additivity (Shannon additivity): if 

rrii n 

Pij > 0, Pi = ^Pij for any i = 1, . . . , n and j = 1, . . . , m», and 2jpi = 1, (7) 

j=l i=l 

then the following equality holds: 

Si (Pll, • • • ,PnmJ = Si Ol, ■ • • ,Pn) + > (—,..., ) , (8) 



Si (pi, . . . , p n ) + "^PiSi (—,..., — J 

i=1 \Pi Pi J 



[SK4] expandability: Si (pi, . . . ,p n , 0) = Si (pi, . . . ,p n ). 

In order to discriminate between the pseudoadditivity (^|) and the above additivity property 
(|), we call the additivity (H) Shannon additivity throughout the paper. 

Until now, two sets of axioms for Tsallis entropy were presented as generalizations of 
the above Shannon-Khinchin axioms 0, [J. In the former axioms presented in [|J, the four 
conditions are given as axioms for Tsallis entropy Sj (p): [Sal] continuity - same as [SKI] 
for Sj, [Sa2] increasing monotonicity - when Pi — - (i = 1, ■ ■ ■ , n), Sj is & monotonic 
increasing function of n for any q G M + , [Sa3] pseudoadditivity - the equality @ holds, 
and [Sa4] generalized Shannon additivity - under the same constraints as (0), the following 
equality holds 



SqiPll,- ■ ■ ,Pnm n ) = Sj(Pl, • • • , Pn) + y^P^SU—, . . . , ^ ) . 

~l \Pi Pi J 



(9) 



This condition (Q) is a generalization of the original one that appeared in Q , which is a case 
of (|D when n = 2. This generalization is a straightforward one [fj. 

In [ffl, the uniqueness theorem stating that the only function satisfying all requirements 
[Sal]~[Sa4] is Tsallis entropy is proved, but actually these four axioms are verbose. In fact, 
it is sufficient to have two axioms, pseudoadditivity [Sa3] and generalized Shannon additivity 
[Sa4], to determine Tsallis entropy uniquely, as has been shown in our paper ||. Thus the 
four axioms [Sal]~[Sa4] in |§ are clearly redundant. 

On the other hand, in the latter axioms presented in ||, the three conditions given as 
axioms for Tsallis entropy Sj are [Abl] continuity and maximality - same conditions as 
[SKI] and [SK2] for Sj, [Ab2] pseudoadditivity using the conditional entropy, and [Ab3] 
expandability - same condition as [SK4] for ST. The uniqueness theorem for Tsallis entropy 
is also proved. 

In this paper, the Shannon-Khinchin axioms are generalized to nonextensive systems and 
the uniqueness theorem for nonextensive entropies including Tsallis entropy is proved rigor- 
ously. In our axioms, Shannon additivity is used as additivity in contrast to pseudoadditivity 
using the conditional entropy in Abe's axioms ||. 

It was found recently that the original Tsallis entropy ([I]) needs to be renormalized to 
satisfy the form invariance of the maximum entropy principle or the pseudoadditivity [TO, IT . 
The normalized Tsallis entropy is defined by 

n 

4 T ( Pl ,..., Pn )EE !=L_. (10) 

i=i 

In order to avoid confusion, we call given by (|IJ) and given by ( |T0D original Tsallis 
entropy and normalized Tsallis entropy respectively. 

In the next section, the axioms for nonextensive entropy including the original Tsallis 
entropy are presented, and the uniqueness theorem is proved rigorously. 

II. GENERALIZED AXIOMS AND UNIQUENESS THEOREM FOR NONEX- 
TENSIVE ENTROPY 



Generalized Shannon-Khinchin axioms and the uniqueness theorem for nonextensive en- 
tropy are given below. 
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Let A n be a n-dimensional simplex defined by @. The following axioms [N1]~[N4] 
determine the function S q : A n — > M + such that 

n 

5 3 (pi,...,p n ) = — ^t4— j ( U ) 

where (g) satisfies properties (i) ~ (iv): 

f > 0, if g > 1 

[ < 0, if < q < 1 

(ii) (g) is differentiable with respect to q G M + , (13) 

(iii) hm^M = l, (14) 



and 



(iv) hiring) = 0(1) =0, 0(g) ^0 (g^l). (15) 
g-+i 



[Nl] S'g is continuous in A n and g G M + , 
[N2] For any g G R + and any n G N, 

S q (p) < S q (-,..-, - J for any p G A n , (16) 

[N3] Under the constraints (|^), the following equality holds: 

S q (pn,---,Pnm n ) = S q (p!,...,p n ) + ^p?5, (—,...,— J , (17) 

i=1 VP* Pi / 

[N4] 

limS'g = S-l = - y^pilnpi. (18) 

q— >1 *— ' 

8=1 

Note that the Shannon-Khinchin axioms and ours differ in the formulation of [N3] and 
[N4] . [N3] is a slight generalization of the Shannon additivity (§) in the sense that instead of 
the exponent "1" of the coefficient pi in front of S q (^-, ■ ■ ■ , ^r" j, any positive real number 
q in p\ can be taken. [N4] shows that Tsallis entropy is a one-parameter generalization of 
the Shannon entropy. [N4] can be replaced by the following expandability: 

[N4*] For any q G 1R + and any (p 1; . . . ,p n ) G A n , 

Sq (Pi, ■ ■ ■ ,Pn,0) = S q (pi, . . . ,p n ) . (19) 



Furthermore, in case of q = 1, the axioms [N1]~[N3] and [N4*] coincide with the Shannon- 
Khinchin axioms |], |7j. Therefore when q = 1 we can obtain Shannon entropy from 
[N1]~[N3] and [N4*]. In the present axioms, for simplicity, [N4] is used instead of [N4*]. 
The proof is given as follows. 

Consider the case of q ^ 1. In the axiom [N3], we apply the following special case: for 
any i = 1, . . . ,n and j = 1, . . . , rrii, if we take 

m = mi = . . . = m n and = , (20) 

mn 

then the identity can be written as 

3.(1. j_w(i,...,i) + £j_*(i,...,iy (21) 

\mn mn J \n n J n q \m m 1 

Let fg {n) be defined by 



n n 



/,(„)=£,(!,..., I), (22) 



so the equation (|2lD becomes 

/, {mn) = f q (n) + n 1 ^/, H . (23) 
Exchanging the variables m and n in (^3|), we have 

/, (n) + n l -if q (m) = /, (m) + m 1 ^/, (n) • (24) 
This can be rearranged to give 



fq i n ) fg M 



(25) 



1 - n 1 -^ 1 - m 1 -^ 

This identity holds for any m, n G N, and it depends on the variable q only Thus, there 
exists a function (q) such that 

1 _ n l-9 

/.(») = -j^p (26) 



where 0(g) is a function of q G R + , satisfying (jT2|)^(|T5|). The properties (|12])~ (|15D are 
required for (ft (q) to satisfy the other axioms [N2] and [N4] . 

If all Pi in (pi, ... , p n ) G A n are rational numbers, then for every pi there exist nonnegative 
integers mi G N satisfying 

V l = -P- (i = 1, • • • , n) . (27) 



i=l 



Here if we take p^ as 



Pij 



8=1 



(28) 



for any i = 1, . . . , n and j = 1, . . . , rrii, then from (|27| ) we have Pi = E Pij- Applying these 
formulas (|27|) and (|^) to (0), the identity (|17D can be reduced to 



/ 



S n 



\ 



n ' ' ' ' ' n 

Y, m i E m * 

i=l i=l 



J 



S q (pi, . . . ,p n ) + y^pfSg ( — , — ) , 

~( \ m i m iJ 



(29) 



i=l 



that is, using f q (n) defined in (p2|), 

/«? ^XJ m *j = S i (Pi> • • • >Pn) + 
Substitution of f q (n) obtained in (|2~6D into (0) yields 

/ n \ n 
(Pi, • • • ,Pn) = fq \ ^2 m i) ~ ^2Pif< 



q \JTli) 



(30) 



iJq 



,i=l 



i=l 



q 1—g 

i=l . i=l \i=l 



1 - E p1 E p? «\ 9 - ( E ™> 
+ 
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The identity (P7|) implies 



so that (|31|) becomes 



(?) 



(?) 



1-9 



(31) 



(32) 



«=i 



i=l 



i-EpI 

Sq( P U...,Pn) = ^ r . 



(33) 



The continuity [Nl] of S q and the fact that any real number can be approximated by rational 
numbers with any precision enable us to validate the above formula for any (pi, . . . , p n ) e A n . 

n 

Here 1 — Ep? * n the right hand side of (|33D is clearly a concave function with respect 

i=l 

to (pi, ■ ■ ■ ,p n ) ^ A n when q > 1, and a convex function when < g < 1. Moreover, 
the obtained entropy ( |33"D is clearly a symmetric function with respect to its arguments 
Pi, . . . ,p n - Thus the axiom [N2] requires the property (|T2| ) for S q (pi, . . . ,p n ) given by (j33|). 



because the property ( |l"2D makes S q (pi, . . . ,p n ) given by (^) a concave function on A n for 
any fixed q G M. + . 



Substitution of ( |3~3"D into [N4] implies 

n 

]M-^- = -£ ft ]ii ft . (34) 
Clearly lim I 1 — J^pf ) = 0, so that the axiom [N4] requires the property (|i~5|) for 0(g). 

V i=l / 

Moreover, the numerator 1 — ^ pf is obviously differentiable with respect to g and its first 



i=l 



derivative is — ^2 Pi m P«- Thus the axiom [N4] requires the property ( |T3"D and (|l"4]) for (g). 



Accordingly, we can apply l'Hdpital's rule to the left hand side of (|34|): 

n 

?-^~^M = "5Z^ ln ^- ( 35 ) 

(iq i=l 

Thus [N4] is satisfied as required. 

Therefore, given a certain function 0, we can uniquely determine nonextensive entropy 
(|TT|). Thus the proof is complete. 



Note that there are many functions 0(g) satisfying all properties (|12[) ~ (13) • For in- 
stance, 

+ 1) (36) 

is one such function, and 

0(g) = ? -l (37) 

is the simplest one. Therefore, the original Tsallis entropy (0) is the simplest of all nonex- 
tensive entropies which can be obtained from the generalized Shannon-Khinchin axioms 
[N1]~[N4]. 

In particular, when (g) = 1 — 2 1_y , S q given by (|ll|) is exactly the same as Havrda- 
Charvat entropy S^ c [H| or Daroczi entropy [|14[| : 

n 

Sf ° ( Pl , . . . , Pn ) = S° (Pi, • • • ,Pn) = 1 ■ (38) 

Thus Havrda-Charvat entropy and Daroczi entropy are also particular examples of the 
nonextensive entropies. Note that 0(g) = 1 — 2 1-9 does not satisfy the condition ( p~4|) 
because of the difference between 2 and e as base of logarithm used in each formulation. 
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It is easy to verify that S q obtained in our uniqueness theorem satisfies the additivity 
property 

S q (A, B) = S q (A) + S q (B) - (q) S q (A) S q (B) . (39) 

When 4>{q) = q — 1, this equality (|39|) takes the form of (|). Thus the form ([39|) is a 
generalization of the pseudoadditivity given by @. In this way the pseudoadditivity (§) or 
(|39l) of the nonextensive entropy can be derived naturally from the slight generalization of 
the Shannon-Khinchin axioms. 



III. CONCLUSION 

There exist two main methods for axiomatic definition of Shannon entropy. The first 
is to introduce inherent properties as axioms for Shannon entropy, as done by Shannon in 
1948 and Khinchin in 1953 @, 0. The other way is to treat satisfactory properties as axioms 
for information content of Shannon entropy, which was used in many references such as 



T2[. This method is based on the fact that Shannon entropy is given by an expectation of 
information content. 

In this paper we present generalized Shannon-Khinchin axioms for nonextensive entropy 
and prove the uniqueness theorem rigorously in accordance with the Shannon-Khinchin 
approach. These results reveal that the Tsallis entropy is the simplest of all nonextensive 
entropies which can be obtained from the generalized Shannon-Khinchin axioms. Moreover, 
the remaining problem such as redundancy in the previously presented axioms in [Q is 
successfully solved. 

On the other hand, the latter way to define nonextensive entropy by means of infor- 



mation content has been already presented in our paper ||11|| . Thus the two manners to 
define nonextensive entropy as similar to Shannon entropy are clearly observed in nonex- 
tensive systems framework. Comparing these two ways to define nonextensive entropy, we 
can point out certain advantages of using information content over the Shannon-Khinchin 



approach. For instance, the latter allows systematic introduction of other entropies [11]. In 
our forthcoming paper, another application of information content will be presented for the 
formalism of Tsallis information theory. 
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